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Evidence for Early Literacy Intervention: 
The Impacts of Reading Recovery 


Introduction 
Henry May 
Research increasingly links low literacy levels in the early grades with a range Philip Sirinides 
of poor outcomes; for instance, students who read below grade level at the air 
end of third grade are about four times less likely than their higher-achieving Abigail Gray 
eers to graduate from high school (Annie E. Casey Foundation, 2010, 2011; 
: : | [ Heather Goldsworthy 


Balfanz, Bridgeland, Bruce & Fox, 2012). In a four-year study, researchers from 

the Consortium for Policy Research in Education (CPRE) at the University of 

Pennsylvania and the Center for Research on Education and Social Policy 

(CRESP) at the University of Delaware examined the effectiveness of Reading 

Recovery—a widely used Ist grade literacy program—at helping struggling Graduate School of Education 
early readers catch up. The study's findings offer promise for intensive early & Penn GSE 
literacy intervention. GS 
Reading Recovery is a literacy intervention for Ist graders. It was developed in 

the 1970s by Marie Clay, a New Zealand psychologist and educator (Clay, 1991; SITY 

2005), and has become one of the most widely used reading programs in the OF 


world. ETAWARE.. 


Reading Recovery consists of a 12- to 20-week series of individual, daily, 
30-minute lessons provided by a specially trained teacher. The program aims to 
help students develop self-directed literacy strategies. 
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EVIDENCE FOR EARLY LITERACY INTERVENTION 


In one of the largest randomized controlled trials ever conducted on an 
instructional program, CPRE and CRESP analyzed data from nearly 7,000 
students between 2011 and 2015. The evaluation was funded by a 2010 
Investing in Innovation (i3) grant to The Ohio State University (OSU)—the 
seat of Reading Recovery in the United States—from the U.S. Department 
of Education's Office of Innovation and Improvement. The grant funded 
the expansion of Reading Recovery to nearly 70,000 additional students, 
and allocated funds for a rigorous, multi-year independent evaluation. 


What the evaluation revealed 


Key findings from the randomized trial component of the CPRE/CRESP evaluation of Reading Recovery 
include the following: 


e Students who received Reading Recovery performed significantly better on a standardized literary assessment 
at the end of the intervention than those who did not. 


e Schools that had the lowest average student achievement tended to have the largest treatment effects, 
suggesting that low-achieving students benefit most from the program. 


e English Language Learners and students in rural schools realized similar benefits to students overall. 
e There was considerable variation from school to school in the size of Reading Recovery’s treatment effects. 


e Findings from an exploratory study of long-term impacts on 3rd grade reading scores were inconclusive as a 
result of the small sample available for the analysis—most students in the study had not yet reached Grade 3. 
Long-term impacts will be addressed in a follow-up study. 


The implementation study component of the evaluation produced a number of insights about schools’ 
use of Reading Recovery. Among these are: 


e Some Reading Recovery lessons are stronger than others, and differences in lesson quality result from both 
teacher and school factors. 


e Schools integrate Reading Recovery in various ways. Those with high levels of principal leadership and 
communication about the program realize the greatest benefits. 


To implement the evaluation, CPRE and CRESP randomly selected a 
group of schools from those participating in the i3-funded scale-up 

of Reading Recovery in each of four years (2011-2015). In the selected 
schools, teachers used assessments to identify the eight lowest-achieving 
lst-grade students in the school. Four of those eight were randomly 
assigned to receive Reading Recovery lessons in addition to regular 
classroom instruction in the first half of the school year (the treatment 
group). The other four served as the control group, receiving regular 
classroom instruction and any other interventions their schools typically 
provide to struggling Ist grade readers. The control group students were 
eligible to receive Reading Recovery later in the year. 
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Students in both treatment and control groups were assessed using 
the lowa Test of Basic Skills (ITBS) Reading Total subtest; ITBS subtests in 
Comprehension and Reading Words; and the Observation Survey of 
Early Literacy Achievement (OS). 


The i3 evaluation also included an in-depth study of Reading Recovery’s 
implementation in schools in the i3 scale-up. This study included 
hundreds of interviews with implementers, including school- and district- 
level administrators; instructional observations; and 23 field-based case 
studies that examined how particular schools incorporated Reading 
Recovery. 


Just how big are the impacts 
of Reading Recovery? 


The i3 evaluation found that, compared with students in the control 
group, Ist graders who were randomlly assigned to Reading Recovery 
scored an average of 3.41 points higher on the ITBS Reading Total scale, 
3.57 points higher on the ITBS Reading Words scale, and 3.90 points 
higher on the ITBS Comprehension scale at the end of the intervention. 
The effect sizes represented by these differences range from 0.30 to 0.42 
standard deviations. 


These treatment effects can be interpreted in several ways. For instance, 
they can be translated to months of student learning. As Figure 1 
illustrates, students who received Reading Recovery made between 
one and two months of additional growth during the roughly five-month 
timeframe of the experiment as compared with the national average 
from the ITBS norming sample. This growth rate is more than 130% of the 
typical average growth rate for Ist grade students. 


Figure 1: Average months of learning for September through 
January of 1st grade 
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The effect sizes for Reading Recovery can also be compared to 
those of other instructional programs. A study by Lipsey et al. 
(2012) presents mean effect sizes for different types of educational 
interventions. For instructional programs like Reading Recovery, 
they report average effect sizes (using Cohen's d) of 0.13. The total 
standardized effect size (Using Cohen's d) for Reading Recovery 
was 0.37. As Figure 2 shows, this indicates that Reading Recovery’s 
effects were 2.8 times greater than the reading outcomes of 
other instructional interventions. Similarly, the impacts of Reading 
Recovery were 3.5 times larger than the average effects of Title | 
programs reviewed by Borman and D'Agostino (1996). 


Figure 2: Average treatment effects for elementary instructional 
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Interventions (Borman & D'Agostino, 1996) 
(Lipsey et al., 2012) 


Key Takeaways 


The CPRE/CRESP evaluation of Reading Recovery was one of the 
largest and most rigorous studies of an educational intervention 
ever conducted. It found treatment effects that are among the 
largest observed from an instructional program. The study therefore 
offers strong evidence of the impact of Reading Recovery on the 
reading skill of struggling Ist graders. In addition, it demonstrates the 
feasibility of effectively scaling up an intervention. 


The study also revealed that some schools’ Reading Recovery 
programs are more effective than others, and that differences in 
teachers’ instruction and in schools’ embrace of the program may 
contribute to variation in school-level impacts. 


The complete final report from the i3 evaluation of Reading 


Recovery is available for download at www.cpre.org/ 
readingrecovery. 
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The making of a Reading 
Recovery teacher 


Reading Recovery takes a unique and intensive approach to 
teacher training. Each first-year Reading Recovery teacher 
participates in a year-long graduate-level course that grounds their 
day-to-day teaching experiences in Reading Recovery theory and 
techniques. 


The course is taught by a highly trained literacy expert who is 
typically an experienced Reading Recovery teacher. In addition 
to teaching the weekly class, the instructor provides side-by-side 
coaching support to teachers in training as they navigate their first 
year, and reduced but ongoing support in successive years. 


One distinctive feature of Reading Recovery training is behind-the- 
glass instruction. In a behind-the-glass session, a teacher conducts 
a real-time lesson with a student behind a two-way mirror, while 

a teacher leader and other teachers observe. After the session, 
observers offer detailed feedback intended to help refine the 
teacher's instruction. 


Reading Recovery teachers’ near-unanimous praise for the training 
process was a key finding early in the i3 study. Comments like 

the following, from a first-year Reading Recovery teacher, were 
common: “! would say | learned more in this one class than | did in 
my entire master’s program to become a reading specialist.” 
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Questions? Find More Info 


Contact CPRE researcher Abigail Gray at grayab@upenn.edu or 
for general questions, email: core@upenn.edu. 


Stay connected and join the conversation online! 


Consortium for Policy Research in Education 
@CPREresearch | CPRE.ORG 


Subscribe to CPRE Insights eNewsletter: bit.ly/CPREinsights 
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The Consortium for Policy Research 

in Education (CPRE) brings together 
education experts from renowned 
research institutions to contribute 

new knowledge that informs PrekK-16 
education policy and practice. Our 
work is peer-reviewed and open-access. 
Our institutions include: 


University of Pennsylvania 

Teachers College, Columbia University 
Harvard University 

Stanford University 

University of Michigan 

University of Wisconsin-Madison 


Northwestern University 


The Consortium for Policy Research in 
Education (CPRE) is headquartered at 
the Graduate School of Education at 
University of Pennsylvania. 


For more information, visit: 
cpre.org. 
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